Stabilization of face in video

ABSTRACT

Placement of a face depicted within a video may be determined. One or more stabilization options for the video may be obtained. Stabilization option(s) may include angle stabilization option, a position stabilization option, and/or a size stabilization option. The video may be stabilized based on the placement of the face and the stabilization option(s).

FIELD

This disclosure relates to stabilization of faces depicted withinvideos.

BACKGROUND

A video may include depiction of a moving thing. A user may wish tomaintain position, angle, and/or scale of the depiction of the movingthing within the video.

SUMMARY

This disclosure relates to stabilizing videos. Video information and/orother information may be obtained. The video information may define avideo having a progress length. The video may include visual content.The visual content may include depiction of a face. Placement of theface within the visual content may be determined as the function ofprogress through the progress length. Selection of one or more facialstabilization options may be obtained. Stabilized visual content may begenerated based on the placement of the face within the visual contentas the function of progress through the progress length, the facialstabilization option(s), and/or other information. The stabilized visualcontent may include stabilized depiction of the face.

A system that stabilizes videos may include one or more electronicstorage, processor, and/or other components. The electronic storage maystore video information, information relating to video, informationrelating to visual content, information relating to depiction of a face,information relating to facial stabilization options, informationrelating to stabilized visual content, and/or other information.

The processor(s) may be configured by machine-readable instructions.Executing the machine-readable instructions may cause the processor(s)to facilitate stabilizing videos. The machine-readable instructions mayinclude one or more computer program components. The computer programcomponents may include one or more of a video information component, aplacement component, a stabilization option component, a generationcomponent, and/or other computer program components.

The video information component may be configured to obtain videoinformation and/or other information. The video information may define avideo. The video may have a progress length. The video may includevisual content and/or other content. The visual content may have a fieldof view. The visual content may include depiction of a face.

The placement component may be configured to determine placement of theface within the visual content. The placement of the face within thevisual content may be determined as the function of progress through theprogress length of the video. In some implementations, the placement ofthe face within the visual content may include one or more of rotation,position, and/or size of the face within the visual content. In someimplementations, the placement of the face within the visual content maybe determined based on a width line that runs across eyes of the face.

The stabilization option component may be configured to obtain selectionof one or more stabilization options. The stabilization option(s) mayinclude one or more facial stabilization options. In someimplementations, the selection of the facial stabilization option(s) maybe obtained based on user input identifying the facial stabilizationoption(s).

In some implementations, the facial stabilization option(s) may includean angle stabilization option, a position stabilization option, a sizestabilization option, and/or other facial stabilization option(s). Theangle stabilization option may reduce angular movement of the stabilizeddepiction of the face within the stabilized visual content. The positionstabilization option may reduce translational movement of the stabilizeddepiction of the face within the stabilized visual content. The sizestabilization option may reduce change in size of the stabilizeddepiction of the face within the stabilized visual content.

The generation component may be configured to generate stabilized visualcontent. The stabilized visual content may be generated based on theplacement of the face within the visual content as the function ofprogress through the progress length, the facial stabilizationoption(s), and/or other information. The stabilized visual content mayinclude the stabilized depiction of the face.

In some implementations, generation of the stabilization visual contentmay include determination of a viewing window for the visual content.The viewing window may be determined as the function of progress throughthe progress length of the video based on the placement of the facewithin the visual content as the function of progress through theprogress length, the facial stabilization option(s), and/or otherinformation. The viewing window may define extents of the visual contentto be included within the stabilized visual content as the function ofprogress through the progress length of the video.

In some implementations, the determination of the viewing window mayinclude determination of rotation of the viewing window with respect tothe field of view of the visual content based on the placement of theface within the visual content and the angle stabilization option. Insome implementations, the determination of the viewing window mayinclude determination of position of the viewing window within the fieldof view of the visual content based on the placement of the face withinthe visual content and the position stabilization option. In someimplementations, the determination of the viewing window may includedetermination of size of the viewing window based on the placement ofthe face within the visual content and the size stabilization option.

These and other objects, features, and characteristics of the systemand/or method disclosed herein, as well as the methods of operation andfunctions of the related elements of structure and the combination ofparts and economies of manufacture, will become more apparent uponconsideration of the following description and the appended claims withreference to the accompanying drawings, all of which form a part of thisspecification, wherein like reference numerals designate correspondingparts in the various figures. It is to be expressly understood, however,that the drawings are for the purpose of illustration and descriptiononly and are not intended as a definition of the limits of theinvention. As used in the specification and in the claims, the singularform of “a,” “an,” and “the” include plural referents unless the contextclearly dictates otherwise.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a system that stabilizes videos.

FIG. 2 illustrates a method for stabilizing videos.

FIG. 3 illustrates an example face depicted within a video.

FIG. 4 illustrates an example viewing window to stabilize depiction of aface within a video.

FIG. 5 illustrates an example viewing window to stabilize depiction of aface within a video.

FIG. 6 illustrates an example viewing window to stabilize depiction of aface within a video.

FIG. 7 illustrates an example viewing window to stabilize depiction of aface within a video.

FIG. 8 illustrates example video frames including depiction of a face.

FIG. 9 illustrates example video frames including stabilized depictionof a face.

FIG. 10 illustrates example feature detection for stabilization.

FIG. 11 illustrates example mirroring of visual content.

DETAILED DESCRIPTION

FIG. 1 illustrates a system 10 that stabilizes videos. The system 10 mayinclude one or more of a processor 11, an interface 12 (e.g., bus,wireless interface), an electronic storage 13, and/or other components.Video information and/or other information may be obtained by theprocessor 11. The video information may define a video having a progresslength. The video may include visual content. The visual content mayinclude depiction of a face. Placement of the face within the visualcontent may be determined by the processor 11 as the function ofprogress through the progress length. Selection of one or more facialstabilization options may be obtained by the processor 11. Stabilizedvisual content may be generated by the processor 11 based on theplacement of the face within the visual content as the function ofprogress through the progress length, the facial stabilizationoption(s), and/or other information. The stabilized visual content mayinclude stabilized depiction of the face.

The electronic storage 13 may be configured to include electronicstorage medium that electronically stores information. The electronicstorage 13 may store software algorithms, information determined by theprocessor 11, information received remotely, and/or other informationthat enables the system 10 to function properly. For example, theelectronic storage 13 may store video information, information relatingto video, information relating to visual content, information relatingto depiction of a face, information relating to facial stabilizationoptions, information relating to stabilized visual content, and/or otherinformation.

Content of one or more videos may be referred to as video content. Videocontent may have a progress length. That is, a video may include videocontent having a progress length. A progress length may be defined interms of time durations and/or frame numbers. For example, video contentof a video may have a time duration of 60 seconds. Video content of avideo may have 1800 video frames. Video content having 1800 video framesmay have a play time duration of 60 seconds when viewed at 30 frames persecond. Other progress lengths, time durations, and frame numbers arecontemplated.

Video content may include visual content, audio content, and/or othercontent. For example, video content may include visual content viewableas a function of progress through the progress length of the videocontent, audio content playable as a function of progress through theprogress length of the video content, and/or other content that may beplayed back as a function of progress through the progress length of thevideo content.

Visual content may refer to content of image(s), video frame(s), and/orvideo(s) that may be consumed visually. For example, visual content maybe included within one or more images and/or one or more video frames ofa video. The video frame(s) may define/contain the visual content of thevideo. That is, video may include video frame(s) that define/contain thevisual content of the video. Video frame(s) may define/contain visualcontent viewable as a function of progress through the progress lengthof the video content. A video frame may include an image of the videocontent at a moment within the progress length of the video. As usedherein, term video frame may be used to refer to one or more of an imageframe, frame of pixels, encoded frame (e.g., I-frame, P-frame, B-frame),and/or other types of video frame. Visual content may be generated basedon light received within a field of view of a single image sensor orwithin fields of view of multiple image sensors.

Visual content (of image(s), of video frame(s), of video(s)) with afield of view may be captured by an image capture device during acapture duration. A field of view of visual content may define a fieldof view of a scene captured within the visual content. A captureduration may be measured/defined in terms of time durations and/or framenumbers. For example, visual content may be captured during a captureduration of 60 seconds, and/or from one point in time to another pointin time. As another example, 1800 images may be captured during acapture duration. If the images are captured at 30 images/second, thenthe capture duration may correspond to 60 seconds. Other capturedurations are contemplated.

Visual content may be stored in one or more formats and/or one or morecontainers. A format may refer to one or more ways in which theinformation defining visual content is arranged/laid out (e.g., fileformat). A container may refer to one or more ways in which informationdefining visual content is arranged/laid out in association with otherinformation (e.g., wrapper format). Information defining visual content(visual information) may be stored within a single file or multiplefiles. For example, visual information defining an image or video framesof a video may be stored within a single file (e.g., image file, videofile), multiple files (e.g., multiple image files, multiple videofiles), a combination of different files, and/or other files. In someimplementations, visual information may be stored within one or morevisual tracks of a video.

Audio content may refer to media content that may be consumed as one ormore sounds. Audio content may include one or more sounds stored in oneor more formats/containers, and/or other audio content. Audio contentmay include one or more sounds captured by one or more sound sensors(e.g., microphone). Audio content may include audio/sound capturedduring the capture of the visual content and/or audio/sound provided asan accompaniment for the visual content. Audio content may include oneor more of voices, activities, songs, music, and/or other audio/sounds.Audio content may include sounds captured by a single sound sensor or anarray of sound sensors. The audio content may be captured by one or moresound sensors of the image capture device (e.g., microphone(s) of and/orcoupled to the image capture device) and/or may be captured by one ormore sound sensors separate from the image capture device (e.g.,microphone(s) separate from the image capture device).

Audio content may be stored in one or more formats and/or one or morecontainers. Information defining audio content (audio information) maybe stored within a single file or multiple files. For example, audioinformation defining sound recorded with frames of a video may be storedwithin a single file (e.g., audio file, video file), multiple files(e.g., multiple audio files, multiple video files), a combination ofdifferent files, and/or other files. Audio information may be storedwith and/or separate from the video information. In someimplementations, audio information may be stored within one or moreaudio tracks of a video.

The processor 11 may be configured to provide information processingcapabilities in the system 10. As such, the processor 11 may compriseone or more of a digital processor, an analog processor, a digitalcircuit designed to process information, a central processing unit, agraphics processing unit, a microcontroller, an analog circuit designedto process information, a state machine, and/or other mechanisms forelectronically processing information. The processor 11 may beconfigured to execute one or more machine-readable instructions 100 tofacilitate stabilizing videos. The machine-readable instructions 100 mayinclude one or more computer program components. The machine-readableinstructions 100 may include one or more of a video informationcomponent 102, a placement component 104, a stabilization optioncomponent 106, a generation component 108, and/or other computer programcomponents.

The video information component 102 may be configured to obtain videoinformation and/or other information. Obtaining video information mayinclude one or more of accessing, acquiring, analyzing, determining,examining, identifying, loading, locating, opening, receiving,retrieving, reviewing, selecting, storing, and/or otherwise obtainingthe video information. The video information component 102 may obtainvideo information from one or more locations. For example, the videoinformation component 102 may obtain video information from a storagelocation, such as the electronic storage 13, electronic storage ofinformation and/or signals generated by one or more sensors, electronicstorage of a device accessible via a network, and/or other locations.The video information component 102 may obtain video information fromone or more hardware components (e.g., an image sensor) and/or one ormore software components (e.g., software running on a computing device).

In some implementations, the video information component 102 may obtainvideo information based on user interaction with a userinterface/application (e.g., video editing application, video playerapplication), and/or other information. For example, a userinterface/application may provide option(s) for a user to stabilize avideo/depiction of a thing within a video. The video informationdefining a video may be obtained based on the user's selection of thevideo through the user interface/video application. Other selections ofvideo for retrieval of video information are contemplated.

The video information may define a video. The video may have a progresslength. The video may include visual content, audio content, and/orother content. The video may include visual content, audio content,and/or other content having a progress length. The video information maydefine a video by including information that defines one or morecontent, qualities, attributes, features, and/or other aspects of thevideo/video content. For example, the video information may define videocontent by including information that makes up the content of the videoand/or information that is used to determine the content of the video.For instance, the video information may include information that makesup and/or is used to determine the arrangement of pixels,characteristics of pixels, values of pixels, and/or other aspects ofpixels that define visual content of the video. The video informationmay include information that makes up and/or is used to determine audiocontent of the video. Other types of video information are contemplated.

In some implementations, the video content may include spherical videocontent. The field of view of the visual content of spherical videocontent may include a spherical field of view. Spherical field of viewmay include full spherical field of view (360 degrees of capture) orpartial spherical field of view (less than 360 degrees of capture). Thevisual content may be viewable from a point of view as the function ofprogress through the progress length. Spherical video content mayinclude and/or may be associated with spatial sounds.

Video information may be stored within a single file or multiple files.For example, video information defining video content may be storedwithin a video file, multiple video files, a combination of differentfiles (e.g., a visual file and an audio file), and/or other files. Videoinformation may be stored in one or more formats or containers. A formatmay refer to one or more ways in which the information defining a videois arranged/laid out (e.g., file format). A container may refer to oneor more ways in which information defining a video is arranged/laid outin association with other information (e.g., wrapper format).

The visual content may have a field of view. The field of view of thevisual content may refer to a portion of a scene that was observed byone or more image capture devices in capturing the visual content. Thefield of view may refer to the extent (e.g., angle, size) of the scenethat is captured within the visual content.

The visual content may include depiction of one or more things. One ormore video frames of a video may include depiction of one or morethings. The visual content may include depiction of a living thingand/or a non-living thing. The visual content may include depiction ofthe entirety of a thing or one or more parts of the thing. For example,the visual content may include depiction of a face (e.g., a person'sface). For instance, the visual content of the video may be captured byan image capture device that includes the face within the field of viewof the image capture device. FIG. 3 illustrates an example face 302depicted within visual content 300 of a video.

The depiction of a thing within the visual content may move during theprogress length of the video. For example, the thing and/or the imagecapture device capturing the video may move during capture of the video.Such movement may cause the depiction of the thing to change placementwithin the field of view of the visual content. Placement of the thingdepicted within the visual content may refer to statement of how thething is depicted within the visual content. Placement of the thingdepicted within the visual content may refer to location and/orarrangement of the depiction of the thing within the visual content.Placement of the thing depicted within the visual content may includeone or more of rotation, position, and/or size of the thing depictedwithin the visual content. For example, a person's head may move duringcapture of the video, which may cause the placement (e.g., rotation,position, size) of the face depicted within the visual content of thevideo to change.

For example, FIGS. 4, 5, 6, and 7 illustrate different placements of aface depicted within different video frames of a video. In FIG. 4 , aface 402 may be depicted at the center of a video frame 400 (at thecenter of the field of view of the video frame 400). The face 402 may berotated to the left. In FIG. 5 , a face 502 may be depicted at aposition that is below and to the left of the center of a video frame500. The face 502 may be upright. The size of the face 502 depictedwithin the video frame 500 may be the same as the size of the face 402depicted within the video frame 400. In FIG. 6 , a face 602 may bedepicted at the center of a video frame 500. The face 602 may beupright. The face 602 depicted in the video frame 600 may be smallerthan in the faces 402, 502 depicted in the video frames 400, 500 (e.g.,due to the face in the scene being farther away from the image capturedevice when the image capture device captured the video frame 600 thanwhen the image capture device captured the video frames 400, 500). InFIG. 7 , a face 702 may be depicted at a position that is above and tothe left of the center of a video frame 700. The face 702 may be rotatedto the right. The size of the face 702 depicted within the video frame700 may be the same size as the face 602 depicted within the video frame600.

The placement component 104 may be configured to determine placement ofthe thing(s) within the visual content. For example, the placementcomponent 104 may be configured to determine placement of a face withinthe visual content. Placement determination of other things within thevisual content is contemplated. Determining placement of a thing withinthe visual content may include ascertaining, detecting, establishing,finding, identifying, obtaining, and/or otherwise determining theplacement of the thing within the visual content. Determining placementof a thing within the visual content may include determining one or moreof rotation, position, and/or size of the thing within the visualcontent.

The placement of the thing(s) within the visual content may bedetermined as the function of progress through the progress length ofthe video (e.g., determining the placement of the thing in differentvideo frames of the video). Determining placement of a thing within thevisual content may include determining changes in the placement of thething as the function of progress through the progress length of thevideo (e.g., determining change in placement in different video framesof the video).

The placement of a thing within the visual content may be determinedbased on user selection, analysis of the visual content, and/or otherinformation. User selection of the placement may refer to user inputthat identifies/designates the particular placement of the thing withinthe visual content. For example, a user may interact with a userinterface to identify one or more of rotation, position, and/or size ofa face within the visual content. Analysis of visual content may includeanalysis of the content of the video frames of the video, such asanalysis of the arrangement of pixels, characteristics of pixels, valuesof pixels, and/or other aspects of pixels of the visual content.Analysis of visual content may include use of computer vision and/orother object detection/recognition techniques to determine the placementof a thing within the visual content. For example, analysis of thecontent of the video frames may be used to determine one or more ofrotation, position, and/or size of a face within the visual content.

In some implementations, the placement of a thing within the visualcontent may be determined based on one or more stable features of thething. A stable feature of a thing may refer to an element/part and/or acombination of elements/parts of the thing that does not change overtime. A stable feature of a thing may refer to an element/part and/or acombination of elements/parts of the thing that does not change as thething moves.

For example, a person's face may include elements/parts that are movedby the muscles of the face. For instance, the shape of a person's mouthmay change (e.g., while talking, yelling, yawning), and the shape of aperson's eyes/eyebrows may change (e.g., while smiling, blinking). Usageof such unstable features to determine the placement of a face mayresult in inaccurate determination of the placement.

Instead, a person's face may include elements/parts that are notmoved/not moved very much (e.g., movement of the element/part is lessthan a threshold movement amount) by the muscles of the face. Forexample, upper cheek points/areas of a person's face may not move/bemoved very much (e.g., change position, change shape) when the person isusing the muscles of the face (e.g., while talking, yelling, yawning,smiling, blinking). As another example, a person's nose and ears may notmove/be moved very much when the person is using the muscles of theface.

Usage of such stable features to determine the placement of a face mayresult in accurate determination of the placement at low cost. Forexample, rather than attempting to determine placement of a face byanalyzing the entirety of the face, the placement of the face may bedetermined by using one or more of left upper cheek point/area, rightupper cheek point/area, nose, ears, and/or other stable features of theface. The placement of the face may be determined by using a combinationof stable elements. For example, the placement of the face within thevisual content may be determined based on a width line that runs acrosseyes of the face. The width line may connect the left upper cheekpoint/area and right upper cheek point/area of the face. The width linemay represent the locations of the left and right upper cheekpoints/areas of the face. The width line/left and right upper cheekpoints/areas of the face may provide indication of the rotation,position, and/or size of the face within the visual content. The widthline/left and right upper cheek points/areas of the face may beresistant to changes/movement of the face.

For example, referring to FIG. 3 , the placement of the face 302 withinthe video frame 300 may be determined based on a width line 316. Thewidth line 316 may connect the left upper cheek point/area 312 and rightupper cheek point/area 314 of the face 302. The position of the widthline 316 (and/or positions of the left upper cheek point/area 312 andright upper cheek point/area 314) may indicate the position of the face302 within the video frame 300. The rotation of the width line 316(and/or orientation of the left upper cheek point/area 312 with respectto the right upper cheek point/area 314) may indicate the rotation ofthe face 302 within the video frame 300. The size of the width line 316(and/or distance between the left upper cheek point/area 312 and rightupper cheek point/area 314) may indicate the size of the face 302 withinthe video frame 300.

In some implementations, the placement of a thing within a particularvideo frame may be determined further based on the placement of thething within one or more prior video frames and/or one or moresubsequent video frames. The placement determined within precedingand/or following video frames may be used limit the placementdetermination within the video frame. Other determination of theplacement of the thing(s) depicted within the visual content iscontemplated.

The stabilization option component 106 may be configured to obtainselection of one or more stabilization options. Obtaining selection ofstabilization option(s) may include one or more of accessing, acquiring,analyzing, determining, examining, identifying, loading, locating,opening, receiving, retrieving, reviewing, selecting, storing, and/orotherwise obtaining the selection of stabilization option(s). Thestabilization option component 106 may obtain selection of stabilizationoption(s) from one or more locations. For example, the stabilizationoption component 106 may obtain selection of stabilization option(s)from a storage location, such as the electronic storage 13, electronicstorage of information and/or signals generated by one or more sensors(e.g., touchscreen display sensor), electronic storage of a deviceaccessible via a network, and/or other locations. The stabilizationoption component 106 may obtain selection of stabilization option(s)from one or more hardware components (e.g., user interface device)and/or one or more software components (e.g., software running on acomputing device).

In some implementations, the stabilization option component 106 mayobtain selection of stabilization option(s) based on user interactionwith a user interface/application (e.g., video editing application,video player application), and/or other information. For example, a userinterface/application may provide option(s) for a user to select optionsto be used in stabilization a video/depiction of a thing within a video.Selection of stabilization option(s) may be obtained based on the user'sselection of the stabilization option(s) through the userinterface/video application. For instance, the userinterface/application may provide individual and/or combination ofstabilization options for selection by the user. The userinterface/application may provide the stabilization options with one ormore video editing tools. For example, different stabilization optionsand/or different combinations of stabilization options may be used fordifferent visual effects/themes, and the user may select thestabilization option(s) by selecting the visual effects/themes providedby/through the user interface/application.

In some implementations, different stabilization options may be selectedand/or provided for selection based on the thing(s) depicted within thevideo. For instance, image/object recognition may be used toclassify/identify the type(s) of thing(s) depicted within the video.Different types of things may be associated with different stabilizationoptions, and the stabilization option(s) associated with the identifiedthing(s) may be provided for selection/selected for use. Otherselections of stabilization option(s) for retrieval are contemplated.

In some implementations, the stabilization option(s) may be selected forthe entire progress length of the video. That is, the stabilizationoption(s) that are selected may be used to stabilize the video/thing(s)depicted within the entire progress length of the video. In someimplementations, the stabilization option(s) may be selected for the oneor more portions of the progress length of the video. That is, thestabilization option(s) that are selected may be used to stabilize thevideo/thing(s) depicted within portion(s) of the progress length of thevideo. In some implementations, different stabilization options may beselected for different portions of the progress length of the video.

A stabilization option may refer to a stabilization technique, approach,and/or criteria that may be chosen for use in stabilizing the thing(s)depicted within the video. A stabilization option may refer to an optionthat allow for control in how the video/thing(s) depicted within thevideo are stabilized. A stabilization option may control/determine how aviewing window is placed within the field of view of the visual contentto generate stabilized visual content. In some implementations, thestabilization options may include one or more facial stabilizationoptions. A facial stabilization option may refer to a stabilizationoption for stabilizing depiction of a face within the video. In someimplementations, the selection of the facial stabilization option(s) maybe obtained based on user input identifying the facial stabilizationoption(s). For example, a user may select particular facialstabilization option(s) for use in stabilizing depiction of a face inthe video. As another example, a use may select an effect and/or a themethat is associated with particular stabilization option(s).

In some implementations, the stabilization option(s) (e.g., facialstabilization options) may include an angle stabilization option, aposition stabilization option, a size stabilization option, and/or otherstabilization option(s). The angle stabilization option may refer to astabilization option that controls/directs stabilization using rotationof a thing depicted within the video. The angle stabilization option mayrefer to a stabilization option that stabilizes the depiction of thething by reducing the extent to which the thing may change rotationalposition (e.g., rotate) within the stabilized video. That is,stabilization of the video using the angle stabilization option mayreduce angular movement (rotation) of the stabilized depiction of thething within the stabilized visual content. For example, using the anglestabilization option for a face depicted within the video may cause thestabilization to reduce angular movement of the face within thestabilized video. The angle stabilization option may control/determinethe rotation of the viewing window within the visual content to offsetthe angular movement of the thing depicted within the visual content.Usage of the angle stabilization option may result in angle correctionin the stabilized visual content.

The position stabilization option may refer to a stabilization optionthat controls/directs stabilization using position of a thing depictedwithin the video. The position stabilization option may refer to astabilization option that stabilizes the depiction of the thing byreducing the extent to which the thing may change translational position(e.g., move up, down, left right) within the stabilized video. That is,stabilization of the video using the position stabilization option mayreduce translational movement of the stabilized depiction of the thingwithin the stabilized visual content. For example, using the positionstabilization option for a face depicted within the video may cause thestabilization to reduce translational movement of the face within thestabilized video. The position stabilization option maycontrol/determine the position of the viewing window within the visualcontent to offset the translational movement of the thing depictedwithin the visual content. Usage of the angle stabilization option mayresult in position correction in the stabilized visual content.

The size stabilization option may refer to a stabilization option thatcontrols/directs stabilization using size of a thing depicted within thevideo. The size stabilization option may refer to a stabilization optionthat stabilizes the depiction of the thing by reducing the extent towhich the thing may change its size within the stabilized video. Thatis, stabilization of the video using the size stabilization option mayreduce change in size of the stabilized depiction of the thing withinthe stabilized visual content. For example, using the size stabilizationoption for a face depicted within the video may cause the stabilizationto reduce change in size of the face within the stabilized video. Otherstabilization options are contemplated. The size stabilization optionmay control/determine the size of the viewing window within the visualcontent to offset the change in size the thing depicted within thevisual content. Usage of the angle stabilization option may result inscale correction in the stabilized visual content.

The stabilization options may be used to stabilize/frame the visualcontent based on placement of thing(s) within the visual content ratherthan based on movement of the image capture device. For example, thestabilization options may be used to generate stabilized visual contentthat focuses on a person (e.g., a person talking/singing, a person in avlogging/selfie video, persons in a video conference). The stabilizationoptions may be used to provide automatically pan and/or zoom effect tokeep the thing(s) within the viewing widow. The stabilization optionsmay be used edit one or more videos to create a consistent view of aperson in the view at different moments within the video. Thestabilization options may be used to stabilize videos in whichinformation on image capture device movement (e.g., IMU data) is notavailable. For example, responsive to such information not beingavailable for a video, the stabilization options may be provided and/orautomatically used to stabilize the video. Other use of thestabilization options are contemplated.

The generation component 108 may be configured to generate stabilizedvisual content. The stabilized visual content may be generated based onthe placement of the thing(s) within the visual content as the functionof progress through the progress length, the stabilization option(s)(selected stabilization option(s)), and/or other information. Thestabilized visual content may include stabilized depiction of thething(s). For example, for visual content including depiction of a face,the generation component 108 may be configured to generate stabilizedvisual zed content based on the placement of the face within the visualcontent as the function of progress through the progress length, thefacial stabilization options (selected facial stabilization option(s)),and/or other information. The stabilized visual content may includestabilized depiction of the face.

The stabilized visual content may include a punchout of one or moreextents (spatial portions) of the visual content within one or moreviewing windows. The stabilized visual content may be generated asimages/video frames including extent(s) of the visual content within theviewing window(s). Inclusion of the extent(s) of the visual contentwithin the viewing window(s) in the stabilized visual content mayeffectuate stabilization of the visual content via selective cropping.The portions of the visual content that are cropped for stabilizationmay depend on the placement of the thing(s) within the visual content asthe function of progress through the progress length and thestabilization option(s).

In some implementations, generation of the stabilization visual contentmay include determination of a viewing window for the visual content.The viewing window may define one or more extents of the visual content.A viewing window may define extents of the visual content to be includedwithin the stabilized visual content as the function of progress throughthe progress length. A viewing window may define which portions of thevisual content are included within the stabilized visual content atdifferent moment within the progress length.

A viewing window may be used to provide a punchout of the visualcontent. A punchout of the visual content may refer to an output of oneor more extents of the visual content for presentation (e.g., currentpresentation, future presentation based on video generated using thepunchout). A punchout of the visual content may refer to extents of thevisual content that is obtained for viewing and/or extraction. Theextents of the visual content viewable/extracted within the viewingwindow may be used to provide views of different spatial portions of thevisual content.

For example, the visual content may include a field of view, and thepunchout of the visual content may include the entire extent of thevisual content (the visual content within the entire field of view) orone or more extents of the visual content (the visual content within oneor more portions of the field of view). The punchout of the visualcontent may be presented on one or more displays, included in one ormore videos, and/or otherwise used for presenting stabilized view of thevisual content.

The viewing window may be determined as the function of progress throughthe progress length of the video based on the placement of the thing(s)(e.g., face) within the visual content as the function of progressthrough the progress length, the stabilization option(s) (e.g., facialstabilization option(s)), and/or other information. The viewing windowmay define extents of the visual content to be included within thestabilized visual content as the function of progress through theprogress length of the video.

One or more of rotation, position, size, shape, and/or othercharacteristics of the viewing window may be determined for differentmoments within the progress length of the visual content. One or more ofrotation, position, size, shape, and/or other characteristics of theviewing window may be determined as the function of progress through theprogress length of the video based on the placement of the thing(s)(e.g., face) within the visual content as the function of progressthrough the progress length, the stabilization option(s) (e.g., facialstabilization option(s)), and/or other information.

For example, determination of the viewing window may includedetermination of a placement of the viewing window within the field ofview of the visual content as the function of progress through theprogress length based on the placement of the thing(s) within the visualcontent as the function of progress through the progress length, thestabilization option(s), and/or other information. The placement of theviewing window may refer to how the viewing window is located within thefield of view of the visual content. The placement of the viewing windowmay include one or more of rotation, position, size, shape, and/or othercharacteristics of the viewing window. The placement of the viewingwindow may be determined to offset changes in the depiction of thething(s) within the visual content, so that the placement of thething(s) depicted within the visual content does not appear tochange/change more than a threshold amount.

In some implementations, the determination of the viewing window mayinclude determination of rotation of the viewing window with respect tothe field of view of the visual content based on the placement of thething(s) (e.g., face) within the visual content and the anglestabilization option. The rotation of the viewing window with respect tothe field of view of the visual content may be determined as a functionof progress through the progress length based on the rotation of thething(s) depicted within the visual content. For example, the rotationof the viewing window with respect to the field of view of the visualcontent may be determined as a function of progress through the progresslength based on the rotation of a face depicted within the visualcontent.

The viewing window may be rotated to compensate for rotation of the facesuch that a view of the visual content within the viewing windowincludes depiction of face that does not appear to rotate/rotate morethan a threshold amount. The rotation of the viewing window may bedetermined for different moments within the progress length. The amountand direction by which the viewing window is rotated may match theamount and direction by which the depicted thing rotates in the visualcontent. For example, based on the face rotating clockwise by 15degrees, the viewing window may be rotated clockwise by 15 degrees.

FIG. 4 illustrates an example viewing window to stabilize depiction of aface within a video. In FIG. 4 , a viewing window 420 may be rotated tothe left (counterclockwise) to compensate for rotation of the face 402within the video frame 400. The rotation of the viewing window 420 maycause the face 402 to appear upright within the viewing window 420, andmay provide angle correction. While the face 402 is corrected to appearupright within the viewing window 420 in FIG. 4 , this is merely as anexample and is not meant to be limiting. In some implementations, it maybe desirable to provide a tilted view of a thing, and the anglecorrection may provide a stabilized view in which the thing is depictedwith a consistent angle.

In some implementations, the determination of the viewing window mayinclude determination of position of the viewing window within the fieldof view of the visual content based on the placement of the thing(s)(e.g., face) within the visual content and the position stabilizationoption. The position of the viewing window within the field of view ofthe visual content may be determined as a function of progress throughthe progress length based on the position of the thing(s) depictedwithin the visual content. For example, the position of the viewingwindow within the field of view of the visual content may be determinedas a function of progress through the progress length based on theposition of a face depicted within the visual content.

The viewing window may be positioned to compensate for change inposition of the face such that a view of the visual content within theviewing window includes depiction of face that does not appear to movetranslationally/move translationally more than a threshold amount. Theposition of the viewing window may be determined for different momentswithin the progress length. The amount and direction by which theviewing window is moved translationally may match the amount anddirection by which the depicted thing moves translationally in thevisual content. For example, based on the face moving to the left by acertain amount, the viewing window may be moved to the left by the sameamount.

FIG. 5 illustrates an example viewing window to stabilize depiction of aface within a video. In FIG. 5 , a viewing window 520 may be positionedso that the center of the viewing window 520 is below and to the left ofthe center of the video frame 500. Such positioning of the viewingwindow 520 may compensate for the translational movement of the face 502within the video frame 500. The positioning of the viewing window 520may cause the face 502 to appear at the center of the viewing window,and may provide position correction. While the face 502 is corrected toappear at the center of the viewing window 520 in FIG. 5 , this ismerely as an example and is not meant to be limiting. In someimplementations, it may be desirable to provide a non-center view of athing, and the position correction may provide a stabilized view inwhich the thing is depicted at a particular non-center location (e.g.,automatically position the face following the rule of thirds).

In some implementations, the determination of the viewing window mayinclude determination of size of the viewing window based on theplacement of the thing(s) (e.g., face) within the visual content and thesize stabilization option. The size of the viewing window may bedetermined as a function of progress through the progress length basedon the size of the thing(s) depicted within the visual content. Forexample, the size of the viewing window may be determined as a functionof progress through the progress length based on the size of a facedepicted within the visual content.

The viewing window may be sized to compensate for change in sized of theface such that a view of the visual content within the viewing windowincludes depiction of face that does not appear to change in size/changein size more than a threshold amount. The size of the viewing window maybe determined for different moments within the progress length. Theamount by which the size of the viewing window is changed and whetherthe size increases or decreases may match the change in size of thething depicted in the visual content. For example, based on the size ofthe face decreasing by 10%, the size of the viewing window may bedecreased by 10%.

In some implementations, the viewing window may be sized based onlargest depiction of the thing(s) within the visual content. The size ofthe viewing window used for the largest depiction of thing(s) maydetermine the scale/zoom factor with which the thing(s) are depictedwithin the viewing window. The size of the viewing window used for thelargest depiction of thing(s) may be the largest size of the viewingwindow for generation of the stabilized visual content. For example, thesize of the viewing window when the largest depiction of a face appearsin the visual content may be used as the reference size for the viewingwindow. As the depiction of the face gets smaller in the viewing window,the size of the viewing window may get smaller to maintain the samescale of the face within the viewing window.

FIG. 6 illustrates an example viewing window to stabilize depiction of aface within a video. In FIG. 6 , a viewing window 620 may be sized(decreased in size) so that the ratio/scale of the face 602 to theviewing window 620 remains consistent with the ratio/scale of the faceto the viewing window at other moments within the progress length of thevideo. For instance, the ratio/scale of the face 602 to the viewingwindow 620 may be the same as/within a threshold amount of theratio/scale of the face 502 to the viewing window 520. The change insize of the viewing window 620 may compensate for change in size of theface 602 within the video frame 600. The sizing of the viewing widow 620may cause the face 602 to appear with same on the display, and mayprovide scale correction. While the face 602 is corrected to appear totake up most of vertical space within the viewing window 620, this ismerely as an example and is not meant to be limiting. Other ratio/scaleof the thing depicted in the visual content to the viewing window iscontempt.

FIG. 7 illustrates an example viewing window to stabilize depiction of aface within a video. In FIG. 7 , a viewing window 720 may be rotated tothe right (clockwise) to compensate for rotation of a face 702 within avideo frame 700. The viewing window 720 may be positioned so that thecenter of the viewing window 720 is above and to the left of the centerof the video frame 500. Such positioning of the viewing window 720 maycompensate for the translational movement of the face 702 within thevideo frame 700. The viewing window 720 may be sized so that theratio/scale of the face 702 to the viewing window 720 remains consistentwith the ratio/scale of the face to the viewing window at other momentswithin the progress length of the video.

FIG. 8 illustrates example video frames 802, 804, 806, 808 includingdepiction of a person, including the person's a face. The video frames802, 804, 806, 808 may include different placement of the face based onmovement of the face during capture. For example, the person may havemoved to the left for capture of the video frame 804, moved to the rightfor capture of the video frame 806, and moved towards the image capturedevice for capture of the video frame 808.

FIG. 9 illustrates example video frames 902, 904, 906, 908 includingstabilized depiction of a face. The video frames 902, 904, 906, 908 mayinclude extents of the video frames 802, 804, 806, 808 within a viewingwindow, with the viewing window placed in the video frames 802, 804,806, 808 to compensate for different placement of the face within thevideo frames 802, 804, 806, 808. The stabilized depiction of the face inthe video frames 902, 904, 906, 908 may not show as much translationalmovement, rotational movement, and/or change in size as the depiction ofthe face in the video frames 802, 804, 806, 808.

FIG. 10 illustrates example feature detection for stabilization. Widthline of a face may be shown in images 1002, 1004, 1006. The image 1002may be an image captured by an image capture device. The image 1004 maybe a punchout of the image 1002, with the viewing window centered on theface. The image 1006 may be a punchout of another image. As can be seenin images 1004, 1006, the punchout of the images may maintain (e.g.,keep same, keep within threshold amount) the rotation, position, andsize of width line, which may result in stabilized view of the face.

In some implementations, the viewing window may be restricted from goingbeyond the field of view of the visual content. That is, the viewingwindow may be required to be placed so that the entirety of the viewingwindow is within the field of view of the visual content. In someimplementations, the viewing window may not be restricted from goingbeyond the field of view of the visual content. That is, the viewingwindow may be allowed to be placed so that one or more portions of theviewing window is outside the field of view of the visual content.Responsive to the viewing window extending beyond the field of view ofthe visual content, one or more extents of the visual content may beduplicated to fill the portion(s) of the viewing window outside thefield of view of the visual content. For example, the visual content maybe mirrored on one or more sides. For instance, FIG. 11 illustratesexample mirroring of visual content. Such duplication of visual contentmay allow the viewing window to extend beyond the original field of viewof the visual content, and still have pixel information to providepunchout for stabilization. For example, video frames 904, 908 in FIG. 9may include mirrored visual content on the left side based on theviewing window extending beyond the field of view of the original visualcontent.

In some implementation, the stabilized visual content may be generatedas an encoded video edit (e.g., an encoded video file). For example, thestabilized visual content may be generated as an encoded in a particularvideo clip, and the video clip may be opened in a video player forpresentation. In some implementations, the stabilized visual content maybe generated as instruction identifying/defining the viewing window tobe used in providing a stabilized view of the visual content. A videoplay use the instructions to retrieve the extents of the video withinthe viewing window for presentation.

Implementations of the disclosure may be made in hardware, firmware,software, or any suitable combination thereof. Aspects of the disclosuremay be implemented as instructions stored on a machine-readable medium,which may be read and executed by one or more processors. Amachine-readable medium may include any mechanism for storing ortransmitting information in a form readable by a machine (e.g., acomputing device). For example, a tangible computer-readable storagemedium may include read-only memory, random access memory, magnetic diskstorage media, optical storage media, flash memory devices, and others,and a machine-readable transmission media may include forms ofpropagated signals, such as carrier waves, infrared signals, digitalsignals, and others. Firmware, software, routines, or instructions maybe described herein in terms of specific exemplary aspects andimplementations of the disclosure, and performing certain actions.

In some implementations, some or all of the functionalities attributedherein to the system 10 may be provided by external resources notincluded in the system 10. External resources may include hosts/sourcesof information, computing, and/or processing and/or other providers ofinformation, computing, and/or processing outside of the system 10.

Although the processor 11 and the electronic storage 13 are shown to beconnected to the interface 12 in FIG. 1 , any communication medium maybe used to facilitate interaction between any components of the system10. One or more components of the system 10 may communicate with eachother through hard-wired communication, wireless communication, or both.For example, one or more components of the system 10 may communicatewith each other through a network. For example, the processor 11 maywirelessly communicate with the electronic storage 13. By way ofnon-limiting example, wireless communication may include one or more ofradio communication, Bluetooth communication, Wi-Fi communication,cellular communication, infrared communication, or other wirelesscommunication. Other types of communications are contemplated by thepresent disclosure.

Although the processor 11 is shown in FIG. 1 as a single entity, this isfor illustrative purposes only. In some implementations, the processor11 may comprise a plurality of processing units. These processing unitsmay be physically located within the same device, or the processor 11may represent processing functionality of a plurality of devicesoperating in coordination. The processor 11 may be configured to executeone or more components by software; hardware; firmware; some combinationof software, hardware, and/or firmware; and/or other mechanisms forconfiguring processing capabilities on the processor 11.

It should be appreciated that although computer components areillustrated in FIG. 1 as being co-located within a single processingunit, in implementations in which processor 11 comprises multipleprocessing units, one or more of computer program components may belocated remotely from the other computer program components. Whilecomputer program components are described as performing or beingconfigured to perform operations, computer program components maycomprise instructions which may program processor 11 and/or system 10 toperform the operation.

While computer program components are described herein as beingimplemented via processor 11 through machine-readable instructions 100,this is merely for ease of reference and is not meant to be limiting. Insome implementations, one or more functions of computer programcomponents described herein may be implemented via hardware (e.g.,dedicated chip, field-programmable gate array) rather than software. Oneor more functions of computer program components described herein may besoftware-implemented, hardware-implemented, or software andhardware-implemented.

The description of the functionality provided by the different computerprogram components described herein is for illustrative purposes, and isnot intended to be limiting, as any of computer program components mayprovide more or less functionality than is described. For example, oneor more of computer program components may be eliminated, and some orall of its functionality may be provided by other computer programcomponents. As another example, processor 11 may be configured toexecute one or more additional computer program components that mayperform some or all of the functionality attributed to one or more ofcomputer program components described herein.

The electronic storage media of the electronic storage 13 may beprovided integrally (i.e., substantially non-removable) with one or morecomponents of the system 10 and/or removable storage that is connectableto one or more components of the system 10 via, for example, a port(e.g., a USB port, a Firewire port, etc.) or a drive (e.g., a diskdrive, etc.). The electronic storage 13 may include one or more ofoptically readable storage media (e.g., optical disks, etc.),magnetically readable storage media (e.g., magnetic tape, magnetic harddrive, floppy drive, etc.), electrical charge-based storage media (e.g.,EPROM, EEPROM, RAM, etc.), solid-state storage media (e.g., flash drive,etc.), and/or other electronically readable storage media. Theelectronic storage 13 may be a separate component within the system 10,or the electronic storage 13 may be provided integrally with one or moreother components of the system 10 (e.g., the processor 11). Although theelectronic storage 13 is shown in FIG. 1 as a single entity, this is forillustrative purposes only. In some implementations, the electronicstorage 13 may comprise a plurality of storage units. These storageunits may be physically located within the same device, or theelectronic storage 13 may represent storage functionality of a pluralityof devices operating in coordination.

FIG. 2 illustrates method 200 for stabilizing videos. The operations ofmethod 200 presented below are intended to be illustrative. In someimplementations, method 200 may be accomplished with one or moreadditional operations not described, and/or without one or more of theoperations discussed. In some implementations, two or more of theoperations may occur simultaneously.

In some implementations, method 200 may be implemented in one or moreprocessing devices (e.g., a digital processor, an analog processor, adigital circuit designed to process information, a central processingunit, a graphics processing unit, a microcontroller, an analog circuitdesigned to process information, a state machine, and/or othermechanisms for electronically processing information). The one or moreprocessing devices may include one or more devices executing some or allof the operation of method 200 in response to instructions storedelectronically on one or more electronic storage mediums. The one ormore processing devices may include one or more devices configuredthrough hardware, firmware, and/or software to be specifically designedfor execution of one or more of the operations of method 200.

Referring to FIG. 2 and method 200, at operation 201, video informationmay be obtained. The video information may define a video having aprogress length. The video may include visual content. The visualcontent may include depiction of a face. In some implementations,operation 201 may be performed by a processor component the same as orsimilar to the video information component 102 (Shown in FIG. 1 anddescribed herein).

At operation 202, placement of the face within the visual content may bedetermined as the function of progress through the progress length. Insome implementations, operation 202 may be performed by a processorcomponent the same as or similar to the placement component 104 (Shownin FIG. 1 and described herein).

At operation 203, selection of one or more facial stabilization optionsmay be obtained. In some implementations, operation 203 may be performedby a processor component the same as or similar to the stabilizationoption component 106 (Shown in FIG. 1 and described herein).

At operation 204, stabilized visual content may be generated based onthe placement of the face within the visual content as the function ofprogress through the progress length and the facial stabilizationoption(s). The stabilized visual content may include stabilizeddepiction of the face. In some implementations, operation 204 may beperformed by a processor component the same as or similar to thegeneration component 108 (Shown in FIG. 1 and described herein).

Although the system(s) and/or method(s) of this disclosure have beendescribed in detail for the purpose of illustration based on what iscurrently considered to be the most practical and preferredimplementations, it is to be understood that such detail is solely forthat purpose and that the disclosure is not limited to the disclosedimplementations, but, on the contrary, is intended to covermodifications and equivalent arrangements that are within the spirit andscope of the appended claims. For example, it is to be understood thatthe present disclosure contemplates that, to the extent possible, one ormore features of any implementation can be combined with one or morefeatures of any other implementation.

What is claimed is:
 1. A system for stabilizing videos, the systemcomprising: one or more physical processors configured bymachine-readable instructions to: obtain video information defining avideo having a progress length, the video including visual content, thevisual content including depiction of a face; determine placement of theface within the visual content as the function of progress through theprogress length based on one or more stable features of the face; andgenerate stabilized visual content based on the placement of the facewithin the visual content as the function of progress through theprogress length, the stabilized visual content including stabilizeddepiction of the face, wherein generation of the stabilization visualcontent based on the placement of the face within the visual content asthe function of progress through the progress length results instabilization of the visual content based on the one or more stablefeatures of the face included within the visual content rather thanmovement of an image capture device that captured the visual content. 2.The system of claim 1, wherein the one or more stable features of theface include one or more elements or one or more parts of the face thatare not moved by muscles of the face.
 3. The system of claim 1, whereinthe one or more stable features of the face include one or more elementsor one or more parts of the face that are not moved more than athreshold amount by muscles of the face.
 4. The system of claim 1,wherein the one or more stable features of the face include a width linethat runs across eyes of the face.
 5. The system of claim 1, wherein thestabilized visual content is generated further based on one or morefacial stabilization options.
 6. The system of claim 5, wherein the oneor more facial stabilization options includes an angle stabilizationoption, a position stabilization option, and/or a size stabilizationoption.
 7. The system of claim 6, wherein the angle stabilization optionreduces angular movement of the stabilized depiction of the face withinthe stabilized visual content.
 8. The system of claim 6, wherein theposition stabilization option reduces translational movement of thestabilized depiction of the face within the stabilized visual content.9. The system of claim 6, wherein the size stabilization option reduceschange in size of the stabilized depiction of the face within thestabilized visual content.
 10. A method for stabilizing videos, themethod performed by a computing system including one or more processors,the method comprising: obtaining, by the computing system, videoinformation defining a video having a progress length, the videoincluding visual content, the visual content including depiction of aface; determining, by the computing system, placement of the face withinthe visual content as the function of progress through the progresslength based on one or more stable features of the face; and generating,by the computing system, stabilized visual content based on theplacement of the face within the visual content as the function ofprogress through the progress length, the stabilized visual contentincluding stabilized depiction of the face, wherein generating thestabilization visual content based on the placement of the face withinthe visual content as the function of progress through the progresslength results in stabilization of the visual content based on the oneor more stable features of the face included within the visual contentrather than movement of an image capture device that captured the visualcontent.
 11. The method of claim 10, wherein the one or more stablefeatures of the face include one or more elements or one or more partsof the face that are not moved by muscles of the face
 12. The method ofclaim 10, wherein the one or more stable features of the face includeone or more elements or one or more parts of the face that are not movedmore than a threshold amount by muscles of the face.
 13. The method ofclaim 10, wherein the one or more stable features of the face include awidth line that runs across eyes of the face.
 14. The method of claim10, wherein the stabilized visual content is generated further based onone or more facial stabilization options.
 15. The method of claim 14,wherein the one or more facial stabilization options includes an anglestabilization option, a position stabilization option, and/or a sizestabilization option.
 16. The method of claim 15, wherein the anglestabilization option reduces angular movement of the stabilizeddepiction of the face within the stabilized visual content.
 17. Themethod of claim 15, wherein the position stabilization option reducestranslational movement of the stabilized depiction of the face withinthe stabilized visual content.
 18. The method of claim 15, wherein thesize stabilization option reduces change in size of the stabilizeddepiction of the face within the stabilized visual content.
 19. A systemfor stabilizing videos, the system comprising: one or more physicalprocessors configured by machine-readable instructions to: obtain videoinformation defining a video having a progress length, the videoincluding visual content, the visual content including depiction of aface; determine placement of the face within the visual content as thefunction of progress through the progress length based on one or morestable features of the face, the one or more stable features of the faceincluding one or more elements or one or more parts of the face that arenot moved or are not moved more than a threshold amount by muscles ofthe face; and generate stabilized visual content based on the placementof the face within the visual content as the function of progressthrough the progress length, the stabilized visual content includingstabilized depiction of the face, wherein generation of thestabilization visual content based on the placement of the face withinthe visual content as the function of progress through the progresslength results in stabilization of the visual content based on the oneor more stable features of the face included within the visual contentrather than movement of an image capture device that captured the visualcontent.
 20. The system of claim 19, wherein the stabilized visualcontent is generated further based on one or more facial stabilizationoptions, the one or more facial stabilization options including an anglestabilization option, a position stabilization option, and/or a sizestabilization option.